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Many complex systems can be described in terms of networks of interacting units. Recent studies 
have shown that a wide class of both natural and artificial nets display a surprisingly widespread 
feature: the presence of highly heterogeneous distributions of links, providing an extraordinary 
source of robustness against perturbations. Although most theories concerning the origin of these 
topologies use growing graphs, here we show that a simple optimization process can also account 
for the observed regularities displayed by most complex nets. Using an evolutionary algorithm 
involving minimization of link density and average distance, four major types of networks are en- 
countered: (a) sparse exponential-like networks, (b) sparse scale-free networks, (c) star networks 
and (d) highly dense networks, apparently defining three major phases. These constraints provide a 
new explanation for scaling of exponent about —3. The evolutionary consequences of these results 
are outlined. 
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Many essential features displayed by complex systems, 
such as memory, stability and homeostasis emerge from 
their underlying network structure |^. Different net- 
works exhibit different features at different levels. Al- 
though some nets, as those present in specific areas of 
the brain cortex, are densely wired |^ most complex 
networks are extremely sparse and exhibit the so-called 
small- world phenomenon Q. An inverse measure of 
sparseness, the so-called network density, is defined as 
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where n is the number of vertices of the network and 
(k) is its average degree. For real networks we have p G 

[io-Mo-i]|]. 

It has been shown that a wide range of real net- 
works can be described by a degree distribution P{k) ~ 
k^^(j){k/£^) where (j){k/S,) introduces a cut-off at some 
characteristic scale i^. Three main classes can be defined 
1^. (a) When ^ is very small, P{k) (/)(/c/^) and thus 
the link distribution is single-scaled. Typically this would 
correspond to exponential or Gaussian distributions; (b) 
as ^ grows, a power law with a sharp cut-off is obtained; 
(c) for large scale- free nets are observed. The last two 
cases have been shown to be widespread and their topo- 
logical properties have immediate consequences for net- 
work robustness and fragility . The three previous sce- 
narios are observed in: (a) power grid systems and neural 
networks , (b) protein interaction maps |^ , metabolic 
pathways jpl and electronic circuits Ml and (c) Internet 
topology Hf, scientific collaborations |[ll|] and [Q lexical 
networks. 

Scale-free nets are particularly relevant due to their 
extremely high homeostasis against random perturba- 
tions and fragility against removal of highly connected 
nodes jl3j. These observations have important conse- 
quences, from evolution to therapy [^j. One possible 




FIG. 1: Density (A), energy (B), clustering coefficient (C) 
and distance (D) as a function of A. Averages over 50 opti- 
mized networks with n = 100, M = (j) , ^ = 2/(2) and p(0) = 
0.2 are shown. A: the optimal network becomes a clique for A 
close to 1. The density of a star network, pstar = 2/n — 0.02 
is shown as reference (dashed line) . The clustering coefficient 
of a Poissonian network C'random = {k)/n is shown as refer- 
ence in C. Notice that Crandom ~ p. The normalized distance 
of a star network is, dstar = 6/(n -|- 1) = 0.059 (dashed line) 
and that of a Poissonian network, drandom ~ logn/log(fc) 
(dotted line) are shown as reference in D. 



explanation for the origin of the observed distributions 
would be the presence of some (decentralized) optimiza- 
tion process. 

Network optimization is actually known to play a 
leading role in explaining allometric scaling in biology 
under the assumption of minimal cost in 
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transportation, the scaling properties of branching struc- 
tures are quite well understood as the outcome of an op- 
timization process. In a related context, local and/or 
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global optimization has been also shown to provide re- 
markable result within the context of channel networks 
pTf . By using optimality criteria linking energy dissi- 
pation and runoff production, the fractal properties in 
the model channel nets were essentially indistinguishable 
from those observed in nature. 

Within the context of complex networks in which the 
only relevant elements are vertices and connections, it 
has been shown that several mechanisms of network evo- 
lution lead to scale-free structures Optimization 
has not been found to be one of them. In a recent study, 
however, it was shown that (Metropolis-based) minimiza- 
tion of both vertex-vertex distance and link length (i.e. 
Euclidean distance between vertices) can lead to the 
small-world phenomenon and hub formation. Nonethe- 
less, this view takes into account Euclidean distance be- 
tween vertices, which is generally not relevant in real 
complex networks. Here we show how minimizing both 
vertex- vertex distance and the number of links leads (un- 
der certain conditions) to the different types of network 
topologies depending on the weight given to each con- 
straint. These two constraints include two relevant as- 
pects of network performance: the cost of physical links 
between units and communication speed among them. 

For the sake of simplicity, we take an undirected graph 
having a fixed number of nodes n and links defined by a 
binary adjacency matrix A — {flij}, 1 < i, j < fi. Given 
a pair of vertices i and j, Uij = 1 if they are linked 
{uij = otherwise) and Dij is the minimum distance be- 
tween them. At time t = 0, we have a randomly wired 
(Poissonian) graph in which two given nodes are con- 
nected with some probability p. The energy function of 
our optimization algorithm is defined as 

E{X)^Xd+{l-X)p 

where < X, d, p < I . A is a parameter controlling the 
linear combination of d and p. The normalized number 
of links, p is defined in terms of 

and it is equivalent to Eq. ^. The normalized vertex- 
vertex distance, d, is defined as d = jjjD™'^^ being 

the average minimum vertex- vertex distance and = 
(n + l)/3 the maximum value of D that can be achieved 
by a connected network. A graph whose adjacency ma- 
trix satisfies — 1 if |i— j| — 1 and — otherwise has 
the maximum average vertex- vertex distance that can be 
achieved by a connected graph of order n. 

The minimization of -E(A) involves the simultaneous 
minimization of distance and number of links (which is 
associated to cost). Notice that minimizing E{X) implies 



connectedness (i.e. finite vertex-vertex distance) except 
for A = 0, where it will be explicitely enforced. 

The minimization algorithm proceeds as follows. At 
time i = 0, the network is set up with a density p(0) 
following a Poissonian distribution of degrees (connect- 
edness is enforced). At time i > 0, the graph is modified 
by randomly changing the state of some pairs of vertices. 
Specifically, with probability /x, each can switch from 
to 1 or from 1 to 0. The new adjacency matrix is ac- 
cepted if E{X,t -I- 1) < E{X,t). Otherwise, we try again 
with a different set of changes. The algorithm stops when 
the modifications on A{t) are not accepted M times in a 
row. Hereafter, ti = 100 (2^, M = (2) 2/(2) H 
and p(0) = 0.2. 

We define the degree entropy on a certain value of A 

as 

n~l 

H = - ^pfelogpfc 

k=l 

where pk is the frequency of vertices having degree k and 
Y^^Zi Pk = 1- This type of informational entropy will be 
used in our characterization of the different phases p^ . 

In Fig. |l| some of the basic average properties dis- 
played by the optimized nets are shown against A. These 
plots, together with the degree entropy in Fig. || suggest 
that four phases are present, separated by three sharp 
transitions at A^ « 0.25, AJ « 0.80 and A3 « 0.95 (see 
arrows in Fig. |I|). The second one separates sparse from 
dense nets and fiuctuations in H{X^) are specially high. 
p(A), C(A) « 1 for A > A* « 0.95. For A = and A = 1 a 
Poissonian and a clique (p(A) ~ 1) network are predicted, 
respectively. 

When taking a more careful look at the sparse domain 
(0, A2), three non-trivial types of networks are obtained 
as A grows: 

a. Exponential networks, i. e. P{k) ^ e^^l^ . 

b. Truncated scale-free networks, i. e. P{k) ~ 
fc-7e-fc/« with 7 = 3.0 and ^ « 20 (for n = 100). 

c. Star network phase (A* < A < Aj) i.e. a central ver- 
tex to which the rest of the vertices are connected 
to (no other connections are possible). Here, 

n — 1 1 

Vk = 4.1 + -4.n-i (2) 

n n 

A star graph has the shortest vertex-vertex dis- 
tance between vertices among all the graphs hav- 
ing a minimal amount of links |24[ . Non- minimal 
densities can be compensated with a decrease in 
distance, so pure star networks are not generally 
obtained. 

The distributions of (a-c) types and that of a dense net- 
work are shown in Figure ^. A detailed examination of 
the transition between degree distributions reveals that 
hub formation explains the emergence of (b) from (a). 
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FIG. 2: Average (over 50 replicas) degree entropy as a function of A with n = 100, M — (2) > M = 2/(2) and p{0) = 0.2. Optimal 
networks for selected values of A are plotted. The entropy of a star network, Hstar = logn — [(n — l)/n] log(n — 1) = 0.056 is 
provided as reference (dashed line). A: an exponential-like network with A = 0.01. B: A scale-free network with A = 0.08. Hubs 
involving multiple connections and a dominance of nodes with one connection can be seen. C: a star network with A = 0.5. B': 
a intermediate graph between B and C in which many hubs can be identified. 
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FIG. 3: Selected cumulative degree distributions of networks 
obtained minimizing E[\. Every distribution is an average 
over 50 optimized networks with n = 100, M = (2)1 M = 
2/(2) and p{Q) = 0.2. A: an exponential-like distribution for 
A = 0.01. B: a power distribution with exponent 7 = 2.0 for 
A = 0.08 (with a sharp cutoff at ^ « 20). C: A = 0.20. D; 
A = 0.50 (almost an star graph). 



hub competition (b') precedes the emergence of a central 
vertex in (c). The emergence of dense graphs from (c) 
consists of a progressive increase in the average degree 



of non-central vertices and a sudden loss of the central 
vertex. The transition to the star net phase is sharp. 
Fig. 1^ shows {H{X)) along with plots of the major types 
of networks. It can be seen that scale-free networks (b) 
are found close to A*. The cumulative exponent of such 
scale-free networks is 7 = 2.0, the same that it would 
be expected for a random network generated with the 
Barabasi- Albert model A necessary condition for 

the emergence of scaling in such model is the use of pref- 
erential attachment when connecting vertices. 

Our scenario suggests that preferential attachment net- 
works might emerge at the boundary between random 
attachment networks (a) and forced attachment (i.e. ev- 
ery vertex connected to a central vertex) networks (c) and 
points that optimization can explain the selection of pref- 
erential attachment strategies in real complex networks. 
In our study, exponential-like distributions appear when 
distance is minimized under high density pressure, in 
agreement with the study by Amaral and co-workers on 
classes of small- world networks This might be the 
case of the power grid and of neural networks |^ . 

Whether or not optimization plays a key role in shap- 
ing the evolution of complex networks, both natural and 
artificial, is an important question. Different mechanisms 
have been suggested to explain the emergence of the 
striking features displayed by complex networks. Most 
mechanisms rely on preferential attachment-related rules, 
but other scenarios have also been suggested [p5l in 
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which external parameters have to be tuned. When deal- 
ing with biological networks, the interplay between emer- 
gent properties derived from network growth and selec- 
tion pressures has to be taken into account. As an ex- 
ample, metabolic networks seem to result from an evo- 
lutionary history in which both preferential attachment 
and optimization are present. The topology displayed 
by metabolic networks is scale-free, and the underlying 
evolutionary history of these nets suggests that prefer- 
ential attachment might have been involved Early 
in the evolution of life, metabolic nets grew by adding 
new metabolites, and the most connected are actually 
known to be the oldest ones. On the other hand, several 
studies have revealed that metabolic pathways have been 
optimized through evolution in a number of ways. This 
suggests that the resulting networks are the outcome of 
both contributions, plus some additional constraints im- 



posed by the available components to the evolving net- 
work |p8| . In this sense, selective pressures might work 
by tuning underlying rules of net construction. This 
view corresponds to Kauffman's suggestion that evolu- 
tion would operate by taking advantage of some robust, 
generic mechanisms of structure formation . 



Acknowledgments 

We acknowledge R. Pastor-Satorras for helpful discus- 
sions and the technical assistance of F. Busquets. This 
work was supported by (and started at) the Santa Fe 
Institute (RFC and RVS) and grants of the Generali- 
tat de Catalunya (FI/2000-00393, RFC) and the CICYT 
(PB97-0693, RVS). 



S. H. Strogatz, Nature 410, 268 (2001). 
S. A. Kauffman, The Origins of Order: Self-Organization 
(Oxford University Press, 1993). 

G. M. Shepherd, ed.. The synaptic organization of the 
bram (Oxford U. Press, New York, 1990). [23] 
D. J. Watts and S. H. Strogatz, Nature 393, 440 (1998). 
Statistics performed on Table I in Ref. [7|. 
L. A. N. Amaral, A. Scala, M. Barthelemy, and H. E. 
Stanley, Proc. Natl. Acad. Sci. 97, 11149 (2000). [24] 
A.-L. Barabasi and R. Albert, cond-mat/0106096 (2001). 

H. Jeong, S. Mason, A.-L. Barabasi, and Z. N. Oltvai, 
Nature 411, 41 (2001). 

H. Jeong, B. Tombor, R. Albert, Z. N. Oltvai, and A.- 
L.Barabasi, Nature 407, 651 (2000). 
R. Ferrer Cancho, C. Janssen, and R. V. Sole, Phys. Rev. 
E 63 (2001). 

M. E. J. Newman, Proc. Natl. Acad. Sci. 98, 404 (2001). 
R. F. i Cancho and R. V. Sole, Proceedings of the Royal 
Society of London B 268, 2261 (2001), santa Fe Institute 
Working Paper 01-03-016. 

R. Albert, H. Jeong, and A.-L. Barabasi, Nature 406, 
378 (2000). 

J. H. B. G. B. West and B. J. Enquist, Science 276 
(1997). 

J. H. Brown and G. B. West, eds., Scaling m Biology 
(Oxford U. Press, New York, 2000). 

J. R. Banavar, A. Maritan, and A. Rinaldo, Nature 399, 
130 (1999). 

L Rodriguez-Rurbe and A. Rinaldo, Fractal River Basins 
(Cambridge U. Press, Cambridge, 1997). 
A.-L. Barabasi and R. Albert, Science 286, 509 (1999). 
N. Mathias and V. Gopal, Phys. Rev. E 63, 1 (2001). [25] 
Higher values of n were very time consuming. The critical 
part of the algorithm is the calculation of d which has cost [26] 
Q{np{^), that is, Q,{n^) and 0{n?). Faster calculation 
implies performing an estimation of d on a random subset [27] 
of vertices or 1st and 2nd neighbours that happened [28] 
to be misleading. 

[21] Intended for expecting that every pair of vertices has [29] 

been allowed to change its state at least once. 
[22] We define the number of changes in the ad- [3Qj 

jacency matrix between generations as c = 



{oijit + l)\i < j and aij(t + 1) ^ aij(t)^. If c = 1 
then d{t + 1) < d{t) and pit -I- 1) = p(t) is impossible. If 
c > 1 then d(t -f 1) < d(t) with p(t + l)= p{t) is allowed. 
Thereafter p is set to enforce i5[c] = 2 > 1. 
Entropy measures of this type have been used in char- 
acterizing optimal channel networks and other models of 
complex systems (see [^^) although they are tipically 
averaged over time. 

It can be shown through induction on n that a star graph 
with a degree distribution 



n — 1 1 

Pk = Ofc.l -I Ofc,n-l 

n n 



(3) 



has the minimum distance possible among all possible 
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dn = ^ Dij 
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